Introduction to the HMM using the example of CAB crisis states

Here, we investigated

• State composition at the group level
• State dynamics at the group level
• Most likely state sequence over time

Same topics in the practical, using data on mood.

Expand the HMM to the multilevel framework

• Enabling individual specific parameters
• Using the example of CAB crisis states

And

• Model selection problem
• Model fit
• Using the example of CAB crisis states

Including a multilevel framework for the HMM, enables simultaneous estimation:

• group level parameter estimates
• individual specific parameter estimates for each individual \(n\)

For the variable & state dependent emission Normal means:

• \(\quad {\mu_{nki} = \bar{\mu}_{ki} + \epsilon_{\left[\mu\right]nki}}\)

For the state dynamics:

• Multinomial logistic regression to estimate transition probabilities \(\gamma_{nij}\)

• \(\gamma_{nij} = \frac{\text{exp}(\alpha_{nij})}{1 + \sum_{{j} = 2}^m \text{exp}(\alpha_{ni{j}})} \quad\) where \(\quad \alpha_{nij} = \bar{\alpha}_{ij} + \epsilon_{\left[\alpha\right]nij}\)

Adopt a Bayesian approach

Individual specific means / intercepts are sampled from a Normal distribution

Combining:

• The prior distribution:
  • Group level distribution denoting overall mean of parameter over individuals, and
  • Variability in the parameter between individuals
• The ‘observed data’:
  • The observed data within a state

26 patients, 60 observations per patient per CAB factor

emiss_subj <- obtain_emiss(CAB_mHMM_4st, level = "individual")

emiss_subj <- obtain_emiss(CAB_mHMM_4st, level = "individual")

Important to check conceptual state equivalence over individuals!

gamma_subj <- obtain_gamma(CAB_mHMM_4st, level = "individual")

Pairing fine grained ESM data with multilevel HMM, we

• Identified four distinct and ascending CAB based crisis states
• Quantified the dynamics of crisis at the group level
• Visualized crisis states over time at the patient individual level
• Observed considerable variation between patients, both in remaining within and transitioning between CAB crisis states
• Highlighted the need for a personalized method

Number of states is a fixed ‘parameter’ in the model, to be determined by the researcher.

• AIC and BIC tend to favor models with (undesirably) large numbers of states.

Pohle & Langrock (2017) suggested a pragmatic step-by-step approach:

• Decide a priori plausible min and max number of states.
• Fit decided on models.
• Inspect state composition of fitted models.
• Use model checking to include / exclude models.
  • In case of Bayesian framework: model convergence!
• Consider model selection criteria for guidance.
• Make a choice, also based on study aim and expert knowledge, and computational considerations.
• When no clear preference, report all (viable) models.

Check: rule out results from a local (instead of global) maximum.

How to check convergence:

1. Fit the multilevel HMM multiple times, using different starting values

\(\rightarrow\) multiple MCMC chains.

• Keep same sensible ordering of \(\bar{\mu}_i\) or \(\bar{\lambda}_i\) given data and process.
• Values of weakly informative prior distributions should be fixed over the chains.

2. Inspect the chains for evidence of non-convergence:

• Visually:
  • Trace plots \(\rightarrow\) parameter estimates over iterations of the MCMC sampler. Chains should not show any trend and good mixing.
  • Density plots \(\rightarrow\) posterior distribution. Should be unimodal.
• Numerically with potential scale reduction factor \(\hat{R}\):
  • Tests for equality of means of the different chains
  • A value of R-hat below 1.2 is used to indicate convergence.

In mixture models, such as HMMs, the orderning of the hidden states do not affect the fit (i.e., likelihood) of the model.

Consequence:

• In Bayesian estimation, the ordering of the states (e.g., state 1: good mood, state 2: bad mood) may switch over the iterations of the estimation procedure

In multilevel HMMs:

• Occurs at the level of the individual estimates. Severe label switching may also affect the group level estimates.

How to check:

• traceplots of the parameters at the individual level

Severe label switching:

• model is not viable.

Minor label switching:

• manually switch back labels of switched parts or
• do not interpret results for individuals with label switching.

Check: does model recover the data correctly on an array of characteristics, aids in revealing model missspecification.

1. Fit a MHMM to the data.
2. Generate simulated data sets based on obtained parameter estimates.
3. For each simulated data set, calculate summary statistics of interest (e.g., mean or variance) and compare to the observed data.
4. Check visually and/or construct an empirical posterior predictive value (PPV)
   • PPV ranges from 0 to 1.
     
   • Good fit: PPV takes intermediate values.
   • Poor fit: PPV takes extreme values (i.e., <0.025 or >0.975).

PPCs can be used to assess model fit at both the individual and group-level.

Check: is the model a good fit to the data of each individual?

• Using the
  • sequence of most likely states for each individual
  • individual specific parameter estimates for the emission distribution
• we predict the most likely outcome at each point in time.
  • E.g., when modelling continuous data \(\rightarrow\) individual and state specific mean of most likely state at each point in time.

Pseudo residuals at each time point: observed - predicted outcome.

Use for typical residual checking: e.g., normality, heteroscedasticity, lack of trend over time, and autocorrelation.

Let’s continue with our own multilevel HMM in the lab!

• Pohle, J., Langrock, R., Van Beest, F. M., & Schmidt, N. M. (2017). Selecting the number of states in hidden Markov models: pragmatic solutions illustrated using animal movement. Journal of Agricultural, Biological and Environmental Statistics, 22, 270-293. DOI: 10.1007/s13253-017-0283-8

1. Obtain forward probabilities \(\alpha_{t(ni)} = Pr\big(O_1 = o_1, ..., O_t = o_t, S_t = i\big)\) for each subject using current values of subject specific model parameters.
2. Sample hidden state sequences in backward manner using \(\alpha_{t(ni)}\) and \(\gamma_{nij}\).
3. Given current subject-specific parameters, draw new group-level parameter estimates using Gibbs step
4. Given
   • observed event sequence for each subject
   • sampled hidden state sequence for each subject
   • group-level parameter distributions
   • draw new subject-specific parameters using RW Metropolis sampler
5. Repeat step 1-4 a very large number of times